Conventional consumer cameras (both film and digital) capture an image having a rectangular imaging area. For cameras using 35 mm film, the horizontal dimension of the imaging area is approximately 36 mm and the vertical dimension of the imaging area is approximately 24 mm.
Many photographers find it very difficult to hold a camera level when capturing a photograph. A photographer often pivots the camera slightly about the optical axis (whereby the optical axis is defined as an axis passing through the center of the image capture frame and the center of the camera lens). These small rotations are generally unintentional and may occur because conventional consumer cameras are lightweight, for example, the popular One Time Use Cameras (OTUCs). Rotations about the optical axis give the appearance that the subjects in the photograph are off-centered, leaning, or tilted.
U.S. Pat. No. 6,011,585 issued Jan. 4, 2000 to Anderson, entitled “Apparatus and Method for Rotating the Display Orientation of a Captured Image” describes a method of determining image format and image orientation from a sensor present in the camera at the time of image capture. Whereas, this sensor can enable one to determine the orientation or format of a captured image, it cannot lead to detecting a small amount of camera rotation. Instead, the sensor identifies the major image orientation (in increments of 90 degrees) by determining which side of an image corresponds with the “top” of the image.
In U.S. Pat. No. 5,227,889, Yoneyama et al describe a video camera having a separate sensor for detecting slant of the camera. The image is then rotated to remove the effect of the slant from the image.
In U.S. Pat. No. 5,528,194, Ohtani et al describe a camera and processing to apply a geometric transform to an image captured with the camera. The geometric transform is derived based on a measured azimuth angle between the camera and the subject. According to col. 4. lines 10-14, the sensor measuring the azimuth needs to record the relative angle to the subject being photographed. The sensor can be a GPS (global positioning satellite) or a geomagnetic sensor that determines a three-dimensional position of both the camera and subject to determine the azimuth angle.
Lutton et al. (in “Contribution to the Determination of Vanishing Points Using Hough Transform,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 16, No. 4, pp. 430-438, April 1994) attempts to detect the vertical direction of an image. The Lutton et al. article teaches one to select the direction that is orthogonal to the most directions in the scene. The implicit assumption is that the scene will contain many horizontal lines. However, this is not always the case. The Lutton method requires a great deal of processing and may be computationally costly.
Some existing systems detect and correct a skewed image. These systems are primarily in the field of document imaging. Nevertheless, documents have a fairly well defined structure, as opposed to more unpredictable consumer type photographs.
U.S. Pat. No. 5,651,075 by Frazier et al., describes a method of compensating for the perspective distortion present in an image of a license plate captured at a known camera to subject distance. This patent describes a method of modifying the perspective by using the known relationship between the camera and the subject, which relationship is always constant. This arrangement would not be practical for general use at reducing perspective distortion.
U.S. Pat. No. 6,400,848 by Gallagher, entitled “A Method for Modifying the Perspective of a Digital Image”, shows a process for compensating from the perspective of image capture. However, this solution cannot lead to compensating for camera rotation about the optical axis. The perspective of image capture relates to tipping the camera in the up and down direction (i.e. about the x-axis), which is much different than rotating a camera about the optical axis. Therefore, U.S. Pat. No. 6,400,848 is limited to describing a method of correction exclusively unique to the problem of camera rotation about the x-axis.
Consequently, a need exists for overcoming the above described drawbacks. More specifically, a need exists for determining slight image orientation changes as an image is captured in a digital camera, so that images stored by the digital camera are corrected for slight changes in image orientation.